Rule Generation Methods Based on Logic Synthesis

One of the most relevant problems in artificial intelligence is allowing a synthetic device to perform inductive reasoning, i.e. to infer a set of rules consistent with a collection of data pertaining to a given real world problem. A variety of approaches, arising in different research areas such as statistics, machine learning, neural networks, etc., have been proposed during the last 50 years to deal with the problem of realizing inductive reasoning. Most of the developed techniques build a black-box device, which has the aim of solving efficiently a specific problem generalizing the information contained in the sample of data at hand without caring about the intelligibility of the solution obtained. This is the case of connectionist models, where the internal parameters of a nonlinear device are adapted by an optimization algorithm to improve its consistency on available examples while increasing prediction accuracy on unknown data. The internal structure of the nonlinear device and the training method employed to optimize the parameters determine different classes of connectionist models: for instance, multilayer perceptron neural networks (Haykin, 1999) consider a combination of sigmoidal basis functions, whose parameters are adjusted by a local optimization algorithm, known as back-propagation. Another example of connectionist model is given by support vector machines (Vapnik, 1998), where replicas of the kernel of a reproducing kernel Hilbert space are properly adapted and combined through a quadratic programming method to realize the desired nonlinear device. Although these models provide a satisfactory way of approaching a general class of problems, the behavior of synthetic devices realized cannot be directly understood, since they generally involve the application of nonlinear operators, whose meaning is not directly comprehensible. Discriminant analysis techniques as well as statistical nonparametric methods (Duda, Hart, & Stork., 2001), like k-nearest-neighbor or projection pursuit, also belong to the class of black-box approaches, since the reasoning followed by probabilistic models to perform a prediction cannot generally be expressed in an intelligible form. However, in many real world applications the comprehension of this predicting task is crucial, since it provides a direct way to analyze the behavior of the artificial device outside the collection of data at our disposal. In these situations the adoption of black-box techniques is not acceptable and a more convenient approach is offered by rule generation methods (Duch, Setiono, & Zurada, 2004), a particular class of machine learning techniques that are able to produce a set of intelligible rules, in the if-then form, underlying the real world problem at hand. Several different rule generation methods have been proposed in the literature: some of them reconstruct the collection of rules by analyzing a connectionist model trained with a specific optimization algorithm (Setiono, 2000; Setnes, 2000); others generate the desired set of rules directly from the given sample of data. This last approach is followed by algorithms that construct decision trees (Hastie, Tibshirani, & Friedman, 2001; Quinlan, 1993) and by techniques in the area of Inductive Logic Programming (Boytcheva, 2002; Quinlan & Cameron-Jones, 1995). A novel methodology, adopting proper algorithms for logic synthesis to generate the set of rules pertaining to a given collection of data (Boros, Hammer, Ibaraki, & Kogan, 1997; Boros et al., 2000; Hong, 1997; Sanchez, Triantaphyllou, Chen, & Liao, 2002; Muselli & Liberati, 2000), has been recently proposed and forms the subject of the present chapter. In particular, the general procedure followed by this class of methods will be outlined in the following sections, analyzing in detail the specific implementation followed by one of these techniques, Hamming Clustering (Muselli & Liberati, 2002), to better comprehend the peculiarities of the rule generation process.